asm qttiufeic qopv 



WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 
H04N 5/45 



Al 



(11) International Publication Number: 
(43) International Publication Date: 



WO 98/27725 

25 June 1998 (25.06.98) 



(21) International Application Number: PCT/US97/22750 

(22) International Filing Date: 10 December 1997 (10.12.97) 



(30) Priority Data: 

08/770,770 



19 December 1996 (19.12.96) US 



(71) Applicant: THOMSON CONSUMER ELECTRONICS, INC. 

[US/US]; 10330 North Meridian Street, Indianapolis, IN 
46290-1024 (US). 

(72) Inventors: RUMREICH, Mark, Francis; 10308 Indian Lake 

Boulevard South, Indianapolis, IN 46236 (US). ZUKAS, 
Mark, Robert; 5725 South Walcott Street Rear, Indianapolis, 
IN 46227 (US). 

(74) Agents: TRIPOLI, Joseph, S. et al.; GE & RCA Licensing 
Management Operation, Inc., P.O. Box 5312, Princeton, NJ 
08543 (US). 



(81) Designated States: AL, AM, AT, AU, AZ, BA, BB, BG, BR, 
BY, CA, CH, CN, CU, CZ, DE, DK, EE, ES, FI, GB, GE, 
GH, HU, ID, IL, IS, JP, KE, KG, KP, KR, KZ, LC, LK, 
LR, LS, LT, LU, LV, MD, MG, MK, MN, MW, MX, NO, 
NZ, PL, FT, RO, RU, SD, SE, SG, SI, SK, TJ, TM, TR, 
TT, UA, UG, UZ, VN, YU, ZW, ARIPO patent (GH, GM, 
KE, LS, MW, SD, SZ, UG, ZW), Eurasian patent (AM, AZ, 
BY, KG, KZ, MD, RU, TJ, TM), European patent (AT, BE, 
CH, DE, DK, ES, FI, FR, GB, GR, IE, IT, LU, MC, NL, 
PT, SE), OAPI patent (BF, BJ, CF, CG, CI, CM, GA, GN, 
ML, MR, NE, SN, TX>, TG). 



Published 

With international search report. 

Before the expiration of the time limit for amending the 
claims and to be republished in the event of the receipt of 
amendments. 



(54) Title: METHOD AND APPARATUS FOR POSITIONING AUXILIARY INFORMATION PROXIMATE AN AUXILIARY IMAGE 
IN A MULTI-IMAGE DISPLAY 



(57) Abstract 

Method and apparatus for 
generating a signal representing 
multi-image video display 
including a main image (200) and 
an auxiliary image (202), e.g., a 
picture-in-picture (PIP) image, 
provides for positioning auxiliary 
information, such as closed caption 
text, proximate the auxiliary image. 
The auxiliary information (208) 
is located within a border region 
(206) for the auxiliary image 
and positioned for indicating to a 
user that the auxiliary information 
is associated with the auxiliary 
image. The region containing 
the auxiliary information moves 
in response to movement of the 
auxiliary image such that the 
auxiliary information remains 
proximate the auxiliary image. 



200 



± 



204 



2311 



XXXXXXXXXXXXXX 



206 



xxxxxxxxxxxxxx 

208 



202 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



AL 


Albania 


ES 


Spam 


LS 


Lesotho 


SI 


Slovenia 


AM 


Armenia 


Fl 


Finland 


LT 


Lithuania 


SK 


Slovakia 


AT 


Austria 


FR 


France 


LU 


Luxembourg 


SN 


Senegal 


AU 


Australia 


GA 


Gabon 


LV 


Latvia 


SZ 


Swaziland 


AZ 


Azerbaijan 


GB 


United Kingdom 


MC 


Monaco 


TD 


Chad 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


MD 


Republic of Moldova 


TG 


Togo 


BB 


Barbados 


GH 


Ghana 


MG 


Madagascar 


TJ 


Tajikistan 


BE 


Belgium 


GN 


Guinea 


MK 


The former Yugoslav 


TM 


Turkmenistan 


BF 


Burkina Faso 


GR 


Greece 




Republic of Macedonia 


TR 


Turkey 


BG 


Bulgaria 


HU 


Hungary 


ML 


Mali 


TT 


Trinidad and Tobago 


BJ 


Benin 


IE 


Ireland 


MN 


Mongolia 


UA 


Ukraine 


BR 


Brazil 


IL 


Israel 


MR 


Mauritania 


UG 


Uganda 


BY 


Belarus 


IS 


Iceland 


MW 


Malawi 


US 


United Slates of America 


CA 


Canada 


IT 


Italy 


MX 


Mexico 


uz 


Uzbekistan 


CF 


Central African Republic 


JP 


Japan 


NE 


Niger 


VN 


Viet Nam 


CG 


Congo 


KE 


Kenya 


NL 


Netherlands 


YU 


Yugoslavia 


CH 


Switzerland 


KG 


Kyrgyzstan 


NO 


Norway 


zw 


Zimbabwe 


CI 


Cote d'lvoire 


KP 


Democratic People's 


NZ 


New Zealand 






CM 


Cameroon 




Republic of Korea 


PL 


Poland 






CN 


China 


KR 


Republic of Korea 


FT 


Portugal 






CU 


Cuba 


KZ 


Kazakstan 


RO 


Romania 






cz 


Czech Republic 


LC 


Saint Lucia 


RU 


Russian Federation 






DE 


Germany 


LI 


Liechtenstein 


SD 


Sudan 






DK 


Denmark 


LK 


Sri Lanka 


SE 


Sweden 






EE 


Estonia 


LR 


Liberia 


SG 


Singapore 







WO 98A27725 



PCT/US97/22750 



1 

METHOD AND APPARATUS FOR POSITIONING AUXILIARY INFORMATION 
PROXIMATE AN AUXILIARY IMAGE IN A MULTI-IMAGE DISPLAY 



5 CROSS-REFERENCE TO RELATED APPLICATIONS 

This application is related to the following commonly-assigned 
U.S. Patent Applications: Serial No. 08/769,329 entitled "TELEVISION 
APPARATUS FOR SIMULTANEOUS DECODING OF AUXILIARY DATA 

10 INCLUDED IN MULTIPLE TELEVISION SIGNALS", Serial No. 08/769,333 
entitled "VIDEO SIGNAL PROCESSING SYSTEM PROVIDING INDEPENDENT 
IMAGE MODIFICATION IN A MULTI_IMAGE DISPLAY", Serial No. 
08/769,331 entitled '''METHOD AND APPARATUS FOR PROVIDING A 
MODULATED SCROLL RATE FOR TEXT DISPLAY", and Serial No. 08/769,332 

1 5 entitled "METHOD AND APPARATUS FOR REFORMATTING AUXILIARY 
INFORMATION INCLUDED IN A TELEVISION SIGNAL", all of which were 
filed in the name of Mark F. Rumreich et al. on the same date as the 
present application. 

2 0 FIELD OF THE INVENTION 

The invention relates to television receivers capable of 

generating a multi-image display having main and auxiliary images such 

as picture-in-picture (PIP) or picture-outside-picture (POP) displays. 

More particularly, the invention relates to a method and apparatus for 
25 displaying auxiliary information, such as closed caption information, 

proximate an auxiliary image in a multi-image display. 

BACKGROUND 

A television signal may include auxiliary information in 
30 addition to video program and audio program information. For example, 
an NTSC (National Television Standards Committee) television signal may 
include two bytes of closed captioning data during the latter half of each 
occurrence of line 21 of field 1. Closed caption data may be decoded and 
displayed to provide a visible text representation of a television 
35 program's audio content. Additional closed caption data and other types 
of similarly encoded auxiliary information, such as extended data services 
information (XDS), may be included in other line intervals such as line 21 
of field 2. United States law requires caption decoders in all television 
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receivers having displays larger than 13 inches and most television 
programming (including video tapes) now includes captioning data. 

Although captioning was developed to aid the hearing 
impaired, captioning can also provide a benefit to non-hearing impaired 
5 viewers as well. Captioning for a multi-image display such as picture-in- 
picture (PIP) or picture-outside-picture (POP) displays is an example of 
this type of additional benefit. For example, activating a PIP feature 
produces an auxiliary image representing the video content of a 
secondary television program signal. The auxiliary image is a small 

10 picture that is inset into a portion of the main picture. However, only the 
audio program associated with the main picture is processed and coupled 
to the speakers of the television. The audio content of the secondary 
signal is lost. Because the audio program is important to the 
comprehension of a television program, the usefulness of a multi-image 

15 display feature such as a PIP display is severely limited by the lack of an 
associated audio program. An approach to solving this problem is to 
display captions, i.e., visible text, representing the PIP audio program in a 
portion of the display. However, the closed caption decoder in most 
television receivers processes only the caption information associated 

20 with the "main" picture, not the small picture signal. 

An exception to this general rule can be found in certain 
television receivers manufactured by Sharp Corporation such as models 
31H-X1200 and 35H-X1200. These Sharp television receivers display 
captions representing the audio of the PIP image by providing a switching 

25 capability that permits coupling the PIP signal to the main caption 

decoder. PIP captions are displayed full size (up to four rows of 32 large 
characters) at the top or bottom of the screen (a user selectable position). 
An example of PIP captioning produced by Sharp television receivers is 
shown in FIG. 1 which depicts a display including main image 100, PIP 

30 image 102 and PIP caption 104. 



SUMMARY OF THE INVENTION 
The invention resides, in part, in the inventors' recognition of 
a number of problems associated with the described PIP captioning 
35 implementation. First, main-picture captioning and small-picture 

captioning cannot be displayed simultaneously. Second, the small image 
combined with the caption display for the small image may obscure the 
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main image to an extent that is objectionable to a user. For example, a 
PIP caption as in the Sharp implementation (up to 20% of the screen area) 
combined with a normal size PIP image (one-ninth of the screen area) 
may obscure more than 30% of the main video display. Third, the small- 
5 picture caption is difficult to follow simultaneously with small-picture 
video because the location of the caption at the top or bottom of the 
screen is physically disconnected from the small picture and may be a 
significant distance from the small picture. Fourth, the appearance of 
small-picture captions is virtually identical to main-picture captions 

10 causing users to become confused as to which image is associated with the 
caption. The combination of these problems may make auxiliary-picture 
captioning that is implemented in the manner described above 
objectionable to an extent that renders auxiliary-picture captioning 
useless for many viewers. 

15 The invention also resides, in part, in providing apparatus and 

a method for solving the described problems associated with the prior art. 
More specifically, the present invention provides for positioning auxiliary 
information, such as closed captioning text characters, that is associated 
with an auxiliary picture in a multi-image display proximate the auxiliary 

20 picture. One aspect of the invention involves combining signals 

representing an auxiliary image, a border region for the auxiliary image, 
and auxiliary information with a signal representing the main image to 
produce a combined signal representing a composite image having the 
auxiliary information within the border region and proximate the 

25 auxiliary image. Another aspect of the invention involves producing a 
signal representing an image having first, second and third regions 
representing a main image, an auxiliary image and auxiliary information, 
respectively, and producing a change in the location of the second region 
such that the third region changes location in response to the change in 

30 location of the second region. Another aspect of the invention involves 
positioning the third region within the image for indicating to a user that 
the auxiliary information is associated with an auxiliary video program 
included in the second region. Another aspect of the invention involves a 
method of generating a multi-image display by combining main and 

35 auxiliary image signals with border and auxiliary information such that 
the auxiliary information is included within a border region and 
proximate the auxiliary image. 
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BRIEF DESCRIPTION OF THE DRAWING 
The teachings of the present invention can be readily 
understood by considering the following detailed description in 
5 conjunction with the accompanying drawings, in which: 

Figure 1 depicts a PIP captioning orientation as implemented 
in the prior art; 

Figure 2 depicts an orientation of auxiliary information 
relative to an auxiliary picture and a main picture in accordance with the 
10 present invention; 

Figure 3 depicts circuitry for generating an exemplary small- 
picture caption in accordance with the present invention; and 

Figures 4 and 5 illustrate various orientations of small-picture 
captioning with respect to a small image and to the main image. 
15 To facilitate understanding, identical reference numerals have 

been used, where possible, to designate identical elements that are 
common to the figures. 

DETAILED DESCRIPTION 

20 For ease of description, the exemplary embodiments depicted 

in the drawing will be described in the context of a picture-in-picture 
(PIP) display system having a small auxiliary picture inset into a large 
main picture. However, the principles of the invention are applicable to 
other multi-image display systems such as a picture-outside-picture 

25 (POP) system in which an auxiliary picture located is located outside of 
,e.g., beside, the main picture. 

FIG. 2 depicts the image orientation of a PIP image 202 in 
relation to a main picture 200 as produced by a PIP captioning image 
generation system of the present invention. The position of the PIP 

30 image 202 within the confines of the main picture 200 is conventionally 
defined by a viewer. Specifically, the viewer, through a remote control, 
defines a vertical line number (vertical position) and a pixel location 
(horizontal position) where one corner (e.g., upper left corner) of the PIP 
image is to be located. The active region 210 of the PIP image 202, where 

35 the PIP video is displayed, has a typical dimension of one third by one 
third of the size of the main picture 200. The PIP image area 210 (active 
region) is circumscribed by a border region 204. This border region is 
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approximately 0.25 inches (0.64 cm) wide. In the normal operating 
mode, e.g., without closed captioning, the border of the PIP image is 
approximately 0.25 inches wide on all sides of the active image area 210. 
Upon activation of the closed captioning for the PIP image, the bottom 
5 border area 206 is extended to a height of approximately 2 inches (5 cm). 
The closed caption information is displayed in this 2 inch wide region 
(referred to as a closed caption window) as two-lines of closed caption 
text 208. The invention provides a method and apparatus for producing 
this extended border area 206 and positioning the closed caption 

10 information 208 within the extended border area 206 (i.e., position the 
caption for the PIP image proximate to the PIP active image area 210). 

Although the depicted embodiment of the display positions 
the closed caption information for the PIP image at the bottom of the PIP 
image area, the PIP closed caption information could as easily be placed 

15 in an extended border area at the top of the PIP image area or anywhere 
else that is proximate the PIP image area 210. 

FIG. 3 depicts circuitry 300 for positioning the PIP closed 
caption information proximate the active PIP image region as depicted in 
FIG. 2. The circuitry contains a main picture timing generator 312 

20 coupled to a multiplexer array 314 and a PIP image generator 302. The 
multiplexer array contains three multiplexers 306, 308 and 310. These 
multiplexers are actively switched, on a pixel-by-pixel basis, to combine 
pixel values (e.g., luminance and color difference signals) and produce the 
images depicted in FIG. 2. Specifically, the third multiplexer 310 inserts 

25 the PIP image border and caption into the main picture; the second 
multiplexer 308 inserts the active PIP video imagery into the border 
region; and the first multiplexer combines closed caption character values 
with border values that forms a PIP captioning window. 

More specifically, the timing generator 312 has as its input a 

30 vertical position 324 and a horizontal position 326 that is user defined for 
locating the PIP image within the boundaries of the main picture. For 
example, a user can determine the location of the PIP image by activating 
a "MOVE" key on a remote control. In a typical application, each 
activation of the MOVE key moves the PIP image to a different corner of 

35 the main display as indicated by the vertical and horizontal position 
values. The system shown in Figure 3 is controlled, for example, by a 
microcomputer (not shown in Figure 3). The microcomputer responds to 
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the user-selected PIP image position by generating two digital values 
representing the horizontal and vertical coordinates of the PIP position. 
The microcomputer stores the digital values in memory and, in a typical 
system, communicates the digital values to the system in Figure 3 via a 
5 data bus to provide vertical position 324 and horizontal position 326. 

In addition to the horizontal and vertical position inputs, 
timing generator 312 receives vertical count 328 and horizontal count 
330 as input signals. These count values indicate the present main 
picture line and pixel. The count values are generated in a conventional 

10 manner by counters (not shown in Figure 3) that count in response to 
timing signals including horizontal and vertical sync. Conventional sync 
signal generation circuitry (not shown in Figure 3) produces the sync 
signals in response to a composite sync component of a television signal. 
In response to the count values, the timing generator 

15 produces three control signals, namely, CAPTION_INSERT, PIP_INSERT 
and FSW (FAST SWITCH). In general, these signals are timing signals that 
are active for certain portions (e.g., a predefined number of pixels) within 
certain lines. For example, the location for the caption within the main 
picture is defined by a number of inclusive lines and pixels. As such, for 

20 all count values that include these lines and pixels, the CAPTION_INSERT 
signal is active to define a rectangular caption window. The beginning of 
the window, e.g., its upper left corner, is defined as an offset of a number 
of lines and pixels from the vertical and horizontal position values (324 
and 326) that define the location of the PIP image. The CAPTION_INSERT 

25 signal is coupled to closed caption generator 304 which generates signal 
INSERT CHARACTER VALUE on path 320 for controlling first multiplexer 
306 as described further below. 

Similarly, the PIP_INSERT and FSW signals are active for 
certain pixels and lines to control insertion of the active PIP image into 

30 the border region as well as insertion of the PIP image with its border 
and captioning into the main picture. Signal PIP_INSERT is also coupled 
to PIP generator 302 for defining where PIP generator 302 should 
position the PIP image pixels relative to the main picture. 

The PIP image generator 302 contains a closed captioned 

35 character generator 304 that produces closed caption characters. Closed 
captioning standard EIA-608 specifies a closed caption character format 
comprising a display character grid of 15 rows by 32 columns with up to 
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four rows of characters being displayed at any one time. Although these 
standard characters could be displayed proximate the image area of the 
PIP image using the present invention, the invention generally uses 
reformatted characters produced by character generator 302. 
5 Reformatting performed by unit 304 comprises translating the standard 
closed caption character set into a reduced character set, utilizing a 
smaller font size, and displaying only two rows of 18 characters each 
within the PIP captioning window, e.g., the two-inch wide border 
extension. The reformatting facilitates viewer comprehension and 

10 minimizes main picture obstruction. One example of a closed captioned 
character generator that provides reformatted characters is disclosed in 
United States Patent Application Serial No. 08/769,332 entitled "METHOD 
AND APPARATUS FOR REFORMATTING AUXILIARY INFORMATION 
INCLUDED IN A TELEVISION SIGNAL" which was filed in the name of 

15 Mark F. Rumreich et al. on the same date as the present application, is 
commonly assigned, and is incorporated herein by reference. 

The PIP generator 302 produces a control signal INSERT 
CHARACTER VALUE on path 320 that is coupled to the control terminal of 
the first multiplexer 306. In addition to the control signal, the PIP 

20 generator produces a PIP picture signal (ACTIVE PIP PIX) that is coupled 
to the second multiplexer 308. Using the PIP generator 302 and its 
accompanying closed caption character generator, the PIP picture or 
image as well as the closed-captioned data is extracted in a conventional 
manner from an auxiliary video signal (AUX VIDEO). Positioning of the 

25 PIP image is controlled by the PIP_INSERT signal that is generated by the 
main picture timing generator 312, e.g., the PIP generator produces the 
PIP image pixels during a period when the PIP_INSERT signal is active. 
Furthermore, the timing generator 312 produces a CAPTION INSERT 
signal that is coupled to the closed-captioned character generator. This 

30 signal controls the position of the closed-captioned window with respect 
to the main picture, e.g., the caption character pixels are positioned at 
pixels and lines where the CAPTION INSERT signal is active. 

The INSERT CHARACTER VALUE control signal (path 320) 
selects as the output of the first multiplexer either a character value 316 

35 (e.g., a white level pixel value) or a border value 318 (e.g., a gray level 
pixel value). The result is an array of character values and border values, 
e.g., white pixels on a gray background, that when taken together as an 
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array of values depict one or more text characters on a gray background. 
The output of the first multiplexer 306 is coupled via path 322 to the first 
input of the second multiplexer 308. The output of the first multiplexer 
is essentially an image (a rectangular border layer) having a constant 
5 luminance value across the entire image except in a region where the 
closed captioned characters are inserted. The characters are located in a 
caption window defined by the CAPTION INSERT signal. 

The second multiplexer 308 combines the active PIP image 
video with the border layer. As such, the second input of multiplexer 308 

10 is the active PIP image video (ACTIVE PIP PIX 332) produced by the PIP 
generator 302. The second multiplexer 308 is controlled by the 
PIP_INSERT signal produced by the timing generator 312. The timing 
generator 312 produces the PIP_INSERT signal to create the active PIP 
image area, e.g., a "high" signal during a number of pixels in each line that 

15 is to contain the PIP pix. 

Specifically, the PIP_INSERT signal selects the first input to 
the second multiplexer for all vertical and horizontal count values outside 
of the active PIP image area. For all vertical and horizontal count values 
within that region, the PIP INSERT signal selects the active PIP image 

20 video for output from the second multiplexer 308. As such, the active 
PIP video is inserted into the border layer proximate the PIP captioning 
window. A similar effect is accomplished if the first and second 
multiplexers are in reverse order, e.g., the active PIP image is combined 
with the border and then multiplexed with the character value. 

25 Timing generator 312 includes conventional logic 

devices comprising, e.g., gates, flip-flops, etc. that generate active states 
on control signals CAPTION_INSERT, FSW, and PIP_INSERT during time 
intervals described above. The specific time intervals utilized in the 
exemplary embodiment are defined by the following relationships 

30 between horizontal count 330 (referred to below as "HC"), vertical count 
328 ("VC"), horizontal position 326 ("HP"), and vertical position 324 
("VP"). Signal CAPTION_INSERT is active (e.g., high or logic 1) when: 

4HP < HC < (4HP + 220); and 
35 , (VP + 75) < VC < (VP + 72 + 18CAP). 
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That is, signal CAPTION_INSERT is active when HC is greater than 4HP and 
less than 4HP + 220, and VC is greater than (VP+75) and less than (VP + 
72 + 18CAP) where "CAP" is a binary value (either 1 or 0) indicating 
whether PIP captioning is enabled. That is, when a user enables PIP 
5 captioning, e.g., by selecting "PIP CAPTIONING ON" from a setup menu, 
CAP has a value of 1. Similarly, signal FSW is active when: 

4HP < HC < (4HP + 232); and 
VP < VC < (VP + 75 + 18CAP); 

10 

Signal PIP_INSERT is active when: 

4HP < HC < (4HP + 22); and 
(VP +3) < VC < (VP + 72). 

15 

Values such as the 4 that is multiplied times HP and the 220 that is added 
to HP define horizontal offsets (e.g., in pixels) that control the horizontal 
position and width of the border, PIP image and PIP caption windows. 
Similarly, values that are added to VP define vertical offsets (e.g., in lines) 

20 that control the vertical position and height of the border, PIP image, and 
PIP caption windows. It will be apparent that these offset values can be 
modified to vary the position and size of the windows as needed. 

Regardless of the ordering of the first and second 
multiplexers, the system provides for keeping the PIP captioning in close 

25 proximity to the PIP image. If the location of the PIP image changes, for 
example, when the user moves the PIP image (such as using the above- 
mentioned "MOVE" key on a remote control), the location of the PIP 
caption moves automatically to remain in close proximity to the PIP 
image. That is, the location of the PIP captioning is determined in 

30 response to the location of the PIP image. Figure 4 illustrates four 

exemplary locations of a PIP image and an exemplary orientation of the 
PIP captioning for each PIP image location. A variation of the 
arrangement of Figure 4 is illustrated in Figure 5 in which PIP captioning 
automatically changes its orientation with respect to the main image and 

35 moves within the border layer. For example, moving the PIP image from 
a top portion of the main image to a bottom portion of the main image 
causes the PIP captioning to move within the border as shown in Figures 
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5A and 5B or as shown in Figures 5C and 5D. Moving the PIP captioning 
within the border layer can, for example, improve readability of the PIP 
captioning and/or minimize interference of the PIP captioning with the 
main image. The particular manner in which the PIP captioning moves 
5 within the border layer can be selected by a user from a setup menu. 

Returning to Figure 3, a third multiplexer 310 selects between 
the PIP image with its border layer and the main picture 334. The third 
multiplexer 310 is driven by a fast switch (FSW) signal generated by the 
timing generator 312. The FSW signal selects the first input to the third 

10 multiplexer 310 (the PIP image and border) for all horizontal and vertical 
count values within the PIP image area including the border region. For 
all vertical and horizontal count values outside of the image and border 
region for the PIP image, the FSW signal selects the main picture. As 
such, the PIP image and its border layer is inserted into the main picture 

15 and the FSW signal defines the width of the border. The signals at the 
output of the multiplexer 310 are coupled to a display driver (not shown 
but well known in the art). 

Using the circuitry of FIG. 3, the display of FIG. 2 is produced. 
The circuitry, in essence, uses a layered approach to image generation. 

20 Specifically, a closed caption text character value is combined with a 

border value to produce a border layer (a gray layer having a predefined 
size and containing closed caption text), then the active PIP pix is 
combined with the border layer, and lastly, the main pix is multiplexed 
with the PIP image, its border and text to create the comprehensive PIP 

25 display of FIG. 2. Because the system provides for locating the closed- 
caption text in close proximity to the PIP image, a viewer can easily 
comprehend the closed caption text in reference to the PIP image. 

Although various embodiments which incorporate the 
teachings of the present invention have been shown and described in 

30 detail herein, those skilled in the art can readily devise many other 

varied embodiments that still incorporate these teachings. For example, 
various configurations of the border region shown in Figure 2 are 
possible. First, various orientations of the border region are possible as 
discussed above and shown in Figures 4 and 5. In addition, the border 

35 region extension containing the auxiliary information can be adjacent to 
the PIP image as shown in Figures 2, 4 and 5 or can be located spaced 
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slightly from the PIP image, e.g., with a region of different color and/or 
brightness. 



V 
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CLAIMS: 

1. Apparatus comprising: 

means for processing (302) a first television signal for 
5 generating a first signal representing auxiliary information included in 
said first television signal; 

means for generating a control signal (312); and 
means responsive to said control signal for combining 
(306,308,310) said first signal, a signal representing an auxiliary image, a 
10 signal representing a border region for said auxiliary image and a second 
television signal representing a main image to produce a combined signal 
representing a composite image including said main image, said auxiliary 
image, said border region and said auxiliary information such that said 
auxiliary information is displayed within said border region and 
15 proximate said auxiliary image. 

2. The apparatus of claim 1 wherein said auxiliary 
information comprises text. 

20 3. The apparatus of claim 2 wherein said text comprises 

closed caption information. 

4. The apparatus of claim 1 wherein 

said means for combining signals comprises a multiplexer; 

25 said control signal generating means comprises a timing 

generator and said control signal comprises a timing signal for causing 
said multiplexer to include said first signal, said signal representing said 
auxiliary image, and said signal representing said border region in said 
combined signal such that said auxiliary information is displayed within 

30 said border region and proximate said auxiliary image. 

5. The apparatus of claim 4 wherein said signal 
representing said border region comprises border values; said auxiliary 
information comprises closed caption information; said multiplexer array 

35 combines said border values with said closed caption information to 

produce a border layer, combines said signal representing said auxiliary 
image with said border layer to produce an intermediate signal, and 
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combines said intermediate signal with said signal representing said main 
image to produce said combined signal. 

6. The apparatus of claim 4 wherein said signal 

5 representing said border region comprises a border value; said auxiliary 
information comprises a closed caption character value; said multiplexer 
comprises: 

a first multiplexer (306) for combining said border value with 
said closed caption character value to produce a border layer 
1 0 representing said border region including said auxiliary information; 

a second multiplexer (308), coupled to said first multiplexer, 
for combining said signal representing said auxiliary image with said 
border layer to produce an intermediate signal; and 

a third multiplexer (310), coupled to said second multiplexer, 
15 for combining said intermediate signal with said signal representing said 
main image. 

7. The apparatus of claim 6 wherein said timing generator 
produces said control signal for said multiplexer in response to a user 

20 defined horizontal and vertical coordinate position (324,326) for said 
auxiliary image, a vertical count value and a horizontal count value, 
where the horizontal and vertical count values (328,330) indicate a 
particular pixel location being displayed in said main image. 

25 8. The apparatus of claim 7 wherein said auxiliary image 

comprises a PIP image or a POP image. 

9. Apparatus comprising: 

means for extracting (302) auxiliary information from an 
30 auxiliary video signal; 

means for processing (306,308,310) a main video signal and 
said auxiliary video signal for generating an output signal representing a 
video image having a first region representing a main video program 
included in said main video signal, having a second region representing 
35 an auxiliary video program included in said auxiliary video signal, and 
having a third region representing said auxiliary information; and 
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means for producing (312) a change in location of said second 
region within said video image; said third region exhibiting a change in 
location within said video image in response to said change in location of 
said second region. 

5 

10. The apparatus of claim 9 wherein said third region 
being located proximate said second region before and after said change 
in location of said second region. 

10 11. The apparatus of claim 10 wherein said second and 

third regions being located in said video image with a first orientation 
before said change in location of said second region; said first orientation 
being maintained following said change in location of said second region. 

15 12. The apparatus of claim 10 wherein said second and 

third regions being located in said video image with a first orientation 
before said change in location of said second region; said first orientation 
changing to a second orientation following said change in location of said 
second region. 

20 

13. Apparatus comprising: 

means for extracting (302) auxiliary information from an 
auxiliary video signal; 

means for processing (306,308,310) a main video signal and 

25 said auxiliary video signal for generating an output signal representing a 
video image having a first region representing a main video program 
included in said main video signal, having a second region representing 
an auxiliary video program included in said auxiliary video signal, and 
having a third region representing said auxiliary information; and 

30 means for positioning (312) said third region in said video 

image for indicating to a user that said auxiliary information is associated 
with said auxiliary video program included in said second region. 

14. The apparatus of claim 13 wherein said means for 
35 positioning said third region also positions said second region in said 

video image; said means for positioning said second and third regions 
being responsive to a user input for producing a change in position of said 
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second and third regions; said third region being positioned subsequent to 
said change in position such that said user associates said auxiliary 
information with said auxiliary video program subsequent to said change 
in position. 

5 

15. The apparatus of claim 14 wherein said second region of 
said video image represents a PIP image or a POP image. 

16. A method for generating a multi-image display having 
10 character information located proximate an auxiliary image within a main 

picture, comprising the steps of: 

extracting (302) auxiliary information from an auxiliary video 
signal representing said auxiliary image; 

combining (306) a border value with said auxiliary 
15 information to produce a border layer containing said auxiliary 
information; 

combining (308) said auxiliary video signal with said border 
layer to produce an intermediate signal representing said auxiliary image, 
a border region and said auxiliary information located proximate said 
20 auxiliary image and within said border region; and 

combining (310) said intermediate signal and a main video 
signal representing said main picture. 
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